{
 "cells": [
  {
   "metadata": {
    "collapsed": true
   },
   "cell_type": "markdown",
   "source": [
    "# 2.2 数据预处理\n",
    "\n",
    "原始数据一般不是张量格式，因此需要使用pandas对数据进行预处理，再转换为张量，才能使用数据。\n",
    "\n",
    "## 2.2.1 读取数据集"
   ],
   "id": "391229aece6a64b9"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-11-15T14:07:13.835246Z",
     "start_time": "2025-11-15T14:07:13.819593Z"
    }
   },
   "cell_type": "code",
   "source": [
    "# 先创建一个数据量用于模拟需要处理的数据\n",
    "import os\n",
    "\n",
    "os.makedirs(os.path.join('..', 'data'), exist_ok=True)\n",
    "data_file = os.path.join('..', 'data', 'house_tiny.csv')\n",
    "with open(data_file, 'w') as f:\n",
    "    f.write('NumRooms,Alley,Price\\n')  # 列名\n",
    "    f.write('NA,Pave,127500\\n')  # 每行表示一个数据样本\n",
    "    f.write('2,NA,106000\\n')\n",
    "    f.write('4,NA,178100\\n')\n",
    "    f.write('NA,NA,140000\\n')"
   ],
   "id": "52c606794eabfae4",
   "outputs": [],
   "execution_count": 2
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-11-15T14:11:58.415923Z",
     "start_time": "2025-11-15T14:11:58.403358Z"
    }
   },
   "cell_type": "code",
   "source": [
    "# 通过pandas读取csv\n",
    "import pandas as pd\n",
    "\n",
    "data = pd.read_csv(data_file)\n",
    "print(data)"
   ],
   "id": "94555ec9911cde6e",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   NumRooms Alley   Price\n",
      "0       NaN  Pave  127500\n",
      "1       2.0   NaN  106000\n",
      "2       4.0   NaN  178100\n",
      "3       NaN   NaN  140000\n"
     ]
    }
   ],
   "execution_count": 4
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "## 2.2.2 处理缺失值\n",
    "\n",
    "NaN表示缺失值（CSV中的NA自动转换为了NaN），为了处理缺失的数据，典型的方法包括插值法和删除法， 其中插值法用一个替代值弥补缺失值，而删除法则直接忽略缺失值。下面使用插值法处理缺失值"
   ],
   "id": "392e8c590815b967"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-11-15T14:20:02.009268Z",
     "start_time": "2025-11-15T14:20:01.993362Z"
    }
   },
   "cell_type": "code",
   "source": [
    "# 分号表示所有行，0:2表示列索引0列到2列（左闭右开）\n",
    "inputs, outputs = data.iloc[:, 0:2], data.iloc[:, 2]\n",
    "inputs = inputs.fillna(inputs.mean()) # 缺失值使用均值填充\n",
    "print(inputs) # 数据框前两列\n",
    "# print(outputs) # 数据框最后一列"
   ],
   "id": "5c9b44b8ed37ac4a",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   NumRooms Alley\n",
      "0       3.0  Pave\n",
      "1       2.0   NaN\n",
      "2       4.0   NaN\n",
      "3       3.0   NaN\n"
     ]
    }
   ],
   "execution_count": 8
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "由于Alley列是离散的，并且可能的取值只用两个Pave和NaN，因此可以将这列拆分为两列，并且使用1和0来表示，这样就将文本转换为了数值",
   "id": "25e61be27bae6ab5"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-11-15T14:22:50.684708Z",
     "start_time": "2025-11-15T14:22:50.671327Z"
    }
   },
   "cell_type": "code",
   "source": [
    "inputs = pd.get_dummies(inputs, dummy_na=True)\n",
    "print(inputs)"
   ],
   "id": "e4762e1e1b015ae8",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   NumRooms  Alley_Pave  Alley_nan\n",
      "0       3.0           1          0\n",
      "1       2.0           0          1\n",
      "2       4.0           0          1\n",
      "3       3.0           0          1\n"
     ]
    }
   ],
   "execution_count": 10
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "## 2.2.3 转换为张量格式\n",
    "\n",
    "通过前面的处理，将数据框所有数据都转换为了数组，从而能进一步转换为张量："
   ],
   "id": "a108a836de8b4d53"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-11-15T14:24:47.370139Z",
     "start_time": "2025-11-15T14:24:46.607473Z"
    }
   },
   "cell_type": "code",
   "source": [
    "import torch\n",
    "\n",
    "# 将pandas数据框转换为numpy张量，再转换为torch张量\n",
    "X = torch.tensor(inputs.to_numpy(dtype=float))\n",
    "y = torch.tensor(outputs.to_numpy(dtype=float))\n",
    "X, y"
   ],
   "id": "9419fefb75fdf1d5",
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(tensor([[3., 1., 0.],\n",
       "         [2., 0., 1.],\n",
       "         [4., 0., 1.],\n",
       "         [3., 0., 1.]], dtype=torch.float64),\n",
       " tensor([127500., 106000., 178100., 140000.], dtype=torch.float64))"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "execution_count": 11
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
